Method for providing an image representation by means of a surgical microscope, and surgical microscope

ABSTRACT

A method for providing an image representation with a surgical microscope, includes capturing a color image representation of a capture region with a camera, capturing a fluorescence image representation of the capture region with a fluorescence camera, generating a detailed image from the captured color image representation with a spatial filter and an edge stop function, mixing the captured color image representation, the captured fluorescence image representation and the generated detailed image to form a mixed image representation, and providing an image signal which encodes the mixed image representation. Further, the disclosure relates to a surgical microscope.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to German patent application DE 10 2021203 187.0, filed Mar. 30, 2021, the entire content of which isincorporated herein by reference.

TECHNICAL FIELD

The disclosure relates to a method for providing an image representationwith a surgical microscope and to a surgical microscope.

BACKGROUND

Distinguishing tumor tissue from non-tumor tissue is one of the mainproblems when performing surgery on tumors, in particular in the regionof the brain. In addition to the information from captured color imagerepresentations (white light image representations), surgicalmicroscopes can also capture fluorescence image representations in thisrespect, wherein fluorescing contrast agents in the tissue are excitedby specific excitation during the capture so that the tumor tissue isrendered visible, and marked, in the captured fluorescence imagerepresentation. In the process, a captured white light imagerepresentation and a captured fluorescence image representation can beoverlaid or mixed. However, disadvantageously, detailed informationitems (e.g., relating to brain vessels or sulci structures in the brain)may be lost.

US 2018/0364470 A1 describes a microscopy system and a microscopy methodfor recording a fluorescence image and a white light image. Such amicroscopy system includes an illumination apparatus for illuminating anobject region and for exciting at least one fluorescent dye, an opticalunit for imaging the object region on at least one fluorescence imagedetector and at least one white light image detector. A beam splitterand a filter are arranged in the beam path provided by the optical unitand are configured such that substantially only the fluorescence emittedby the fluorescence dyes is incident on the fluorescence image detectorand an image that is as color neutral as possible is recorded by thewhite light image detector.

US 2016/0007856 A1 describes a fluorescence observation apparatus,including a light source configured to irradiate an object with whitelight and excitation light; and a processor including hardware, theprocessor being configured to implement the following: a fluorescenceimage production unit and a white light image production unit which areconfigured to produce a fluorescence image and a color white lightimage, respectively; a state adjustment unit which is configured toadjust weights on an individual basis for a plurality of color componentimages that form the white light image; and a combination unit which isconfigured to combine at least one color component image to which thefluorescence image has been added, and the other color component imageswhile the weights are applied, and the condition adjustment unit whichis configured to adjust the weights on the basis of the color componentimages so that the weight for the color component image to which thefluorescence image is added is larger than the weight for the othercolor component images.

SUMMARY

It is an object of the disclosure to provide a method for providing animage representation with a surgical microscope, and a surgicalmicroscope, in the case of which the mixing of a color imagerepresentation and a fluorescence image representation is improved, andin the case of which details in particular are maintained in the mixedimage representation.

The object is achieved by a method for providing an image representationwith a surgical microscope and a surgical microscope as describedherein.

It is a general concept of the disclosure to generate a detailed imagefrom a captured color image representation, which may also be referredto as white light image representation or RGB image representation inparticular, said detailed image being additionally mixed into the mixedimage representation. In this context, the detailed image is generatedfrom the captured color image representation with a spatial filter, inparticular a two-dimensional spatial filter, and an edge stop function.This renders it possible to extract relevant details from the colorimage representation and additionally consider these when mixing thecaptured color image representation and the captured fluorescence imagerepresentation. Then, the extracted details are still easy to identifyin the mixed image representation.

In particular, a method for providing an image representation with asurgical microscope is provided, the method including: capturing a colorimage representation of a capture region with a camera, capturing afluorescence image representation of the capture region with afluorescence camera, generating a detailed image from the captured colorimage representation with a spatial filter and an edge stop function,mixing the captured color image representation, the capturedfluorescence image representation and the generated detailed image toform a mixed image representation, and providing an image signal whichencodes the mixed image representation.

Further, a surgical microscope in particular is provided, including acamera which is configured to capture a color image representation of acapture region, a fluorescence camera which is configured to capture afluorescence image representation of the capture region, and anevaluation device, the evaluation device being configured to extract adetailed image from the captured color image representation with aspatial filter and an edge stop function, to mix the captured colorimage representation, the captured fluorescence image representation andthe extracted detailed image to form a mixed image representation, andto provide an image signal which encodes the mixed image representation.

The method and the surgical microscope are advantageous in that detailsof the captured color image representation still are contained even inthe case of an image representation that has been mixed from thecaptured color image representation and the captured fluorescence imagerepresentation. In particular, the method and the surgical microscopeallow the details to be highlighted in the mixed image representation.In particular, vessels and sulci structures in the brain can thus stillbe rendered clearly and easily visible in the mixed image representationeven after mixing. A surgeon and/or assistants can register thesedetails better as a result and information present in the captured colorimage representation and in the captured fluorescence imagerepresentation can be optimally represented. An operational sequence anda workflow during surgery can be improved as a result.

The surgical microscope in particular includes a light source forilluminating the capture region, in which an object to be captured, inparticular a body part of a patient, is arranged, with light, inparticular white or broadband light. Further, the surgical microscopemay also include an (additional) excitation light source for exciting afluorescence dye. The surgical microscope may further include opticalelements, in particular for focusing and/or magnification purposes. Theoptical elements may also be part of the camera(s). Further, a beamsplitter may be arranged in a beam path of the surgical microscope inorder, for example, to guide light from the capture region to both thecamera and the fluorescence camera. Further, optical filters may also beused when capturing fluorescence image representations.

In particular, provision is made for the spatial filter to be configuredas a smoothing filter without creating additional artifacts in theprocess. In this case, the edge stop function is used in particular toprevent or reduce excessive smoothing effects at edges. In particular,the detailed image is produced by virtue of the captured color imagebeing filtered with the spatial filter, in particular the smoothingfilter, while taking account of the edge stop function. The detailedimage arises in particular from a difference between the captured colorimage representation and the color image representation filtered in thisway.

In particular, the camera is in the form of a color image detector orwhite light image detector, that is to say that the camera is configuredin particular to capture light in the wavelength range of white light,for example in the wavelength range of visible light. The cameraproduces and in particular provides a signal which represents anintensity distribution of the light incident on the camera from thecapture region, that is to say a color image representation (or whitelight image representation or RGB image representation). In particular,the color image representation is polychromatic. The camera may includeoptical elements (lenses, mirrors, beam splitters, optical filters,etc.) for focusing and/or magnification purposes and/or for beamguidance.

In particular, the fluorescence camera is in the form of thefluorescence image detector, that is to say that the fluorescence camerais configured to capture light in the emission wavelength range of atleast one fluorescence dye in the capture region with spatialresolution. The fluorescence camera produces and in particular providesa signal which represents an intensity distribution of the lightincident on the fluorescence camera, that is to say a fluorescence imagerepresentation. In particular, the fluorescence image representation ismonochromatic. The fluorescence camera may include optical elements(lenses, mirrors, beam splitters, optical filters, etc.) for focusingand/or magnification purposes and/or for beam guidance and/or filtering.

During mixing, provision can be made for monochromatic image informationto be converted into a polychromatic color space or a polychromaticcolor model, for example if a monochromatic image information itemshould be mixed to a polychromatic image information item.

The image signal may have both an analog form and a digital form. Inparticular, the image signal may also be provided in the form of adigital data packet which is saved in a volatile or nonvolatile memoryor storage medium and/or which is output via an interface configured tothis end.

The mixed image representation or the image signal can be displayed on adisplay device.

Parts of the surgical microscope, in particular the evaluation device,can be designed, either individually or together, as a combination ofhardware and software, for example as program code that is executed on amicrocontroller or microprocessor. However, provision can also be madefor parts to be designed as application-specific integrated circuits(ASICs) and/or field-programmable gate arrays (FPGAs), eitherindividually or together. In particular, the evaluation device mayinclude at least one computing device and at least one memory.

In an exemplary embodiment, provision is made for an intensity image tobe produced from the captured color image representation, the detailedimage being produced from the produced intensity image. As a result, acomputing power required for the production of the detailed image can bereduced, and so producing the detailed image can be carried outparticularly efficiently. By way of example, provision can be made forthe captured color image representation (RGB image representation) to beconverted into the YCbCr color space and for subsequently only theluminance channel Y, which corresponds to the intensity in particular,to be considered. This can be done with the following MATLAB code, forexample:

img_ycbcr=rgb2ycbcr(img_rgb); % Convert RGB image representation toYCbCr

img_Y=img_ycbcr(:,:,1);% Extract intensity information

However, other intensity representations may also be used as a matter ofprinciple. By way of example, provision can alternatively also be madeof processing at least one color channel of the color imagerepresentation (RGB image representation) on an individual basis and ofusing this as intensity information. If a plurality of color channelsare processed on an individual basis, a detailed image can be producedfor each of the color channels. The produced detailed images cansubsequently be merged, for example mixed, and can continue to be usedin mixed fashion in the method.

In an exemplary embodiment, provision is made for the intensity image tobe produced and/or processed on a logarithmic scale. As a result, thespatial filter in particular can be implemented particularly efficientlyin relation to a required computing power. Building on the MATLAB codespecified above, the conversion into the logarithmic representation canbe implemented with the following MATLAB code, for example:

img_Y_log=log 10(double(img_Y)/255); % Convert into log representation

In an exemplary embodiment, provision is made for the spatial filter tobe a two-dimensional Gaussian spatial filter. The two-dimensionalGaussian spatial filter produces a smoothing effect without producingadditional artifacts in the process.

In an exemplary embodiment, provision is made for a position-relatedvalue of a gradient of an intensity in the produced intensity image tobe used in each case as input parameter of the edge stop function. Inparticular, such a gradient of the intensity is determined for eachposition of the intensity image, that is to say for each pixel orpixel-by-pixel. To this end, the following measure for example can beused, where x and y each denote coordinates of the considered pixel:

∥∇I(x,y)∥=√{square root over (∇_(x) ²(x,y)+∇_(y) ²(x,y))}

In an exemplary embodiment, provision is made for the detailed image tobe produced by weighted summation of the captured color imagerepresentation filtered with the spatial filter and the captured colorimage representation, followed by a subtraction of the summation imagerepresentation from the captured color image representation, with anoutput value of the edge stop function being used as weightingparameter. In this way, the details are extracted from the color imageand provided as detailed image. By way of example, it is possible todefine the following edge stop function:

$w_{r} = \left\{ \begin{matrix}{\left( {1 - \left( \frac{x}{\lambda} \right)^{2}} \right)^{2},\left( {x \leq \lambda} \right)} \\{0,\left( {x > \lambda} \right)}\end{matrix} \right.$

Here, x is the value of the gradient that was determined with theaforementioned measure, in particular at the respective pixel. In thiscase, the parameter λ allows the sensitivity of the edge stop functionto be adjusted, by way of example λ=0.6.

The summation is subsequently carried out:

(1−w _(r))·l _(f) +w _(r) ·l _(n|)

Here, l_(f) is the intensity value from the captured color imagerepresentation and l_(n|) is the intensity value from the captured colorimage representation filtered with the spatial filter, in particular thetwo-dimensional spatial filter, more particularly the two-dimensionalGaussian spatial filter. The summation is implemented pixel-by-pixel inparticular, where the weighting parameter w_(r) is chosen or calculatedfor the respectively considered pixel (in particular with thecoordinates x, y; see above).

The summation image representation obtained hereby is subtracted fromthe captured color image representation, in particular pixel-by-pixel.If the intensity is used, this is carried out in particular using theintensity image and the intensity image filtered with the spatialfilter.

In an exemplary embodiment, provision is made for the production of thedetailed image to be repeated iteratively with altered parameters. Inparticular, this allows the iterative use of different parameters, forexample in order to obtain certain target criteria. In particular, theindividual iteration steps are independent of one another in this case.However, at least one termination condition is monitored during theiteration. By way of example, the iteration is terminated if thefollowing termination condition for the gradient of the intensity of thecaptured color image representation is satisfied:

∥∇l _(f) ∥<k|l _(n) −l|

because otherwise a smoothing effect of the spatial filter may becometoo strong. Independent of this termination condition, provision can bemade for the iteration to be terminated if a certain number ofiterations have been run through without the termination condition beingsatisfied (e.g., after 12 iterations, etc.).

In principle, there are three options for the possible implementation ofthe mixing of the captured color image representation, the capturedfluorescence image representation and the produced detailed image toform a mixed image representation.

In the first option, the captured color image representation and thecaptured fluorescence image representation are typically mixed first andthe produced detailed image is subsequently added. In particular, themonochromatic fluorescence image representation can be converted into apolychromatic color space or a polychromatic color model before mixing.Alternatively, it is also possible to assign fixed colors orpolychromatic colors (on the basis of an intensity distribution).

In the second option, the produced detailed image is typically added tothe captured color image first and the color image representationenriched in this way is subsequently mixed with the fluorescence imagerepresentation. Especially when using an intensity image for producingthe detailed image, it is advantageous to convert the produced(monochromatic) detailed image into a polychromatic color space or apolychromatic color model prior to mixing with the captured color image.

In an exemplary embodiment, which represents the third option, provisionis made, during mixing, for the produced detailed image to be mixed withthe fluorescence image representation and for the resultant enrichedfluorescence image representation to be mixed with the captured colorimage representation. If the detailed image was produced from anintensity image, the information in the produced detailed image islocated within a monochromatic color space (intensity image). This meansthat the produced detailed image is compatible with the fluorescenceimage representation which is also monochromatic. If the produceddetailed image is (initially) mixed with the fluorescence imagerepresentation, it is possible as a result to avoid errors andartifacts, in particular at edges, which would be visible in thenon-monochromatic color space. This can improve quality of the mixedimage representation.

In an exemplary embodiment, provision is made for a geometric distortioncorrection and/or a shading correction to be carried out for the cameraand the fluorescence camera. As a result, the captured color imagerepresentation and the captured fluorescence image representation can bealigned and/or scaled with respect to one another such that arespectively contained image content is overlaid in the mixed imagerepresentation. In particular, defects possibly present in a beam pathand in an imaging optical unit of the camera and of the fluorescencecamera can be compensated (electronically) hereby if this cannot beimplemented, or is not implemented, by optical methods or opticalelements.

In a further exemplary embodiment, provision is made for an imageimprovement and/or an image enrichment to be carried out. By way ofexample, a measure for improving the contrast may be implemented.Likewise, the fluorescence image representation may be color coded, as aresult of which the information from the fluorescence imagerepresentation is better identifiable in a mixed image representation.To this end, there may also be color coding on the basis of theintensity values (“pseudo-coloring”). An image enrichment may furtherinclude the identification and marking of a boundary of a tumor in thecaptured fluorescence image representation.

In an exemplary embodiment, provision is made for the method to becarried out on a stereoscopic camera and/or stereoscopic fluorescencecamera. As a result, it is possible to produce and provide astereoscopic mixed image representation, which allows a surgeon and/oran assistant to stereoscopically register the capture region. The methodmay be carried out accordingly for each of the channels (left andright).

Further features relating to the configuration of the surgicalmicroscope arise from the description of configurations of the method.Here, the advantages of the surgical microscope are respectively thesame as in the configurations of the method. In particular, theevaluation device of the surgical microscope is configured to carry outthe additional features of the exemplary embodiments of the method.

Further, a method for providing an image representation for a surgicalmicroscope is also provided, the method including: receiving a colorimage representation of a capture region captured with a camera,receiving a fluorescence image representation of the capture regioncaptured with a fluorescence camera, producing a detailed image from thereceived color image representation with a spatial filter and an edgestop function, mixing the received color image representation, thereceived fluorescence image representation and the produced detailedimage to form a mixed image representation, and providing an imagesignal which encodes the mixed image representation. The method can becarried out with a data processing device, including in particular atleast one computing device and at least one memory.

Further features for configuring the method for providing an imagerepresentation for a surgical microscope emerge from the description ofconfigurations of the method for providing an image representation witha surgical microscope. The advantages of the method are the same in eachcase.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure will now be described with reference to the drawingswherein:

FIG. 1 shows a schematic illustration of the surgical microscopeaccording to an exemplary embodiment of the disclosure;

FIG. 2 shows a schematic illustration of a flowchart of the method forthe purposes of explaining a method of processing the captured colorimage representation and the captured fluorescence image representationwith the evaluation device according to an exemplary embodiment of thedisclosure;

FIG. 3 shows a schematic block diagram of the method for providing animage representation with a surgical microscope according to anexemplary embodiment of the disclosure;

FIG. 4 shows a schematic block diagram for elucidating one exemplaryembodiment of the method; and

FIG. 5 shows a schematic illustration for elucidating a processing chainin a surgical microscope and the integration of the method described inthis disclosure into this processing chain.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

FIG. 1 shows a schematic illustration of a surgical microscope 1according to an exemplary embodiment of the disclosure. The surgicalmicroscope 1 includes a camera 2, a fluorescence camera 3 and anevaluation device 4. The surgical microscope 1 is configured to carryout the method described in this disclosure for providing an imagerepresentation 20.

The camera 2 is configured to capture a color image representation 11 ofthe capture region 10. The fluorescence camera 3 is configured tocapture a fluorescence image representation 12 of the capture region 10.The regions captured by the camera 2 and the fluorescence camera 3 mayin particular also have different sizes provided that a common captureregion 10 is present. In particular, the fluorescence camera 3 issensitive to a wavelength range in which a suitable fluorescence dyeemits. In particular, a body part of a patient to be captured (notshown) is arranged in the capture region 10.

The surgical microscope 1 may further include at least one illuminationdevice (not shown), the latter serving to illuminate the capture region10 and/or the targeted excitation of a fluorescence dye.

In particular, the evaluation device 4 includes at least one computingdevice 5, for example a microprocessor or a microcontroller, and atleast one memory 6. To process the captured color image representation11 and the captured fluorescence image representation 12, the computingdevice 5 executes program code, in particular program code stored in thememory 6.

The captured color image representation 11 and the captured fluorescenceimage representation 12 are supplied to the evaluation device 4 by wayof interfaces 7 that are accordingly configured to this end.

The evaluation device 4 is configured to produce or extract a detailedimage 13 from the captured color image representation 11 with a spatialfilter and an edge stop function. In particular, the spatial filter is atwo-dimensional Gaussian spatial filter.

Further, the evaluation device 4 is configured to mix the captured colorimage representation 11, the captured fluorescence image representation12 and the produced or extracted detailed image 13 to form a mixed imagerepresentation 20.

In this case, provision can be made for the detailed image 13 to beproduced by weighted summation of the captured color imagerepresentation 11 filtered with the spatial filter and the capturedcolor image representation 11, followed by a subtraction of thesummation image representation 11 s from the captured color imagerepresentation 11, with an output value of the edge stop function beingused as weighting parameter.

The mixed image representation 20 is provided in the form of an imagesignal 21 which encodes the mixed image representation 20. To this end,the evaluation device 4 produces the image signal 21, for example in theform of an analog or digital image signal 21, in particular in the formof a digital data packet. The produced image signal 21 is provided at aninterface 7 configured to this end.

The provided image signal 21 can subsequently be output, in particulardisplayed, on a display device 8 such that a surgeon and/or an assistantis/are able to register the mixed image representation 20.

Provision can be made for the surgical microscope 1 to be in the form ofa stereoscopic surgical microscope 1, that is to say it respectively hastwo channels for capturing the color image representation 11 and thefluorescence image representation 12, for example as described in US2018/0364470 A1. Then, the method is carried out for each channel suchthat a separate mixed image representation 20 and a separate imagesignal 21 is produced and provided for each of the two channels.Provision can be made for a stereoscopic image representation or astereoscopic image signal (not illustrated) to be produced therefrom.

FIG. 2 shows a schematic illustration of a flowchart of an exemplaryembodiment of the method, which shows a method of processing thecaptured color image representation 11 and the captured fluorescenceimage representation 12 with the evaluation device.

Provision can be made for the evaluation device to be configured toproduce an intensity image 14 from the captured color imagerepresentation 11, the detailed image 13 being produced from theproduced intensity image 14. By way of example, to this end provisioncan be made for the captured color image representation 11 to beconverted into the YCbCr color model and for only the luminanceinformation (i.e., Y) to be subsequently used as intensity information.However, it is also possible in principle to use other intensity values(e.g., also individual color channels of the captured color imagerepresentation 11, etc.).

In particular, provision can be made for a logarithmic scale orrepresentation to be used to represent the intensity information fromthe intensity image 14.

The intensity image 14 is subsequently filtered with a spatial filter15, which is a two-dimensional spatial filter in particular. Inparticular, the two-dimensional spatial filter 15 is a two-dimensionalGaussian spatial filter.

A weighting parameter 17 is determined with an edge stop function 16:

$w_{r} = \left\{ \begin{matrix}{\left( {1 - \left( \frac{x}{\lambda} \right)^{2}} \right)^{2},\left( {x \leq \lambda} \right)} \\{0,\left( {x > \lambda} \right)}\end{matrix} \right.$

Here, x is the value of an intensity gradient that is determined withthe measure set forth below, in particular at the respective pixel. Inthis case, the parameter λ allows the sensitivity of the edge stopfunction 16 to be adjusted, by way of example λ=0.6.

A gradient of the intensity in the intensity image 14 is supplied to theedge stop function 16 via the input value x, in particularpixel-by-pixel (with coordinates x and y):

∥∇I(x,y)∥=√{square root over (∇_(x) ²(x,y)+∇_(y) ²(x,y))}

Subsequently, the captured color image representation 11 or theintensity image 14 and the captured color image representation 11spatially filtered with the spatial filter 15 or the spatially filteredintensity image 14 are summated or mixed with pixel-by-pixel weightingsto form a summation image representation 11 s:

(1−w _(r))·l _(f) +w _(r) ·l _(n|)

Here, l_(f) is the intensity value from the intensity image 14 of thecaptured color image representation 11 and l_(n|) is the intensity valuefrom the captured color image representation 11 filtered with thespatial filter 15, in particular the two-dimensional spatial filter,more particularly the two-dimensional Gaussian spatial filter, or fromthe intensity image 14. In particular, the summation is implementedpixel-by-pixel, the weighting parameter 17 (i.e., w_(r)) being chosen orcalculated for the respectively considered pixel.

The summation image representation 11 s is subtracted from the capturedcolor image representation 11 or the intensity image 14, the resultyielding the detailed image 13.

The detailed image 13 and the captured fluorescence image representation12 are mixed to form an enriched fluorescence image representation 12 a.The captured color image representation 11 and the enriched fluorescenceimage representation 12 a are subsequently mixed, as a result of whichthe mixed image representation 20 is produced. In particular, themonochromatic enriched fluorescence image representation 12 a isconverted into a suitable color space or into a suitable color model inthe process so that mixing with the captured color image representation11 is rendered possible.

FIG. 3 shows a schematic block diagram of an exemplary embodiment of themethod for providing an image representation with a surgical microscope.By way of example, the surgical microscope is a surgical microscopeaccording to the exemplary embodiment shown in FIG. 1.

A color image representation and a fluorescence image representation ofa capture region are captured with a camera and with a fluorescencecamera, respectively, in a measure 100. By way of example, such acapture region may include a tumor in the brain of a patient, whichtumor should be operated on and has been marked with a fluorescence dye.

A detailed image is generated from the captured color imagerepresentation with a spatial filter and an edge stop function in ameasure 101. In particular, a two-dimensional Gaussian spatial filtercan be used to this end.

The captured color image representation, the captured fluorescence imagerepresentation and the generated detailed image are mixed to form amixed image representation in a measure 102.

An image signal which encodes the mixed image representation isprovided, in particular generated and/or output, in a measure 103.

In a measure 104, provision can be made for the mixed imagerepresentation to be displayed on a display device by virtue of theimage signal being supplied to a display device.

In a measure 100 a, provision can be made for a geometric distortioncorrection and/or a shading correction to be carried out for the cameraand the fluorescence camera. In particular, the corrections are carriedout on the captured color image representation and the capturedfluorescence image representation.

In a measure 100 b, provision can be made for an intensity image to begenerated from the captured (optionally corrected) color imagerepresentation, the detailed image being generated from the generatedintensity image in measure 101. In particular, provision can be made forthe intensity image to be generated and/or processed on a logarithmicscale or in a logarithmic representation.

In measure 101, provision can be made for a position-related value of agradient of an intensity in the produced intensity image to be used ineach case as input parameter of the edge stop function.

In a development, provision can be made in measure 101 for the detailedimage to be produced by weighted summation of the captured color imagerepresentation filtered with the spatial filter and the captured colorimage representation, followed by a subtraction of the summation imagerepresentation from the captured color image representation, with anoutput value of the edge stop function being used as weightingparameter.

Further, provision can be made for the production of the detailed imageto be repeated iteratively with altered parameters. To this end, measure101 is repeated until at least one termination criterion has beensatisfied or a predetermined number of iterations has been carried out.In this respect, whether the termination criterion has been satisfied orthe predetermined number of iterations has been reached is monitored ina measure 101 a.

In measure 102, provision can be made, during mixing, for the produceddetailed image to be mixed with the fluorescence image representationand for the resultant enriched fluorescence image representation to bemixed with the captured color image representation.

Provision can be made for the method to be carried out on a stereoscopiccamera and/or stereoscopic fluorescence camera. To this end, measures100 to 104 are carried out for each channel of the stereoscopic surgicalmicroscope. As a result, it is possible to make a stereoscopic mixedimage representation available and for example display the latter inmeasure 104.

FIG. 4 shows a schematic block diagram for elucidating an exemplaryembodiment of the method for providing an image representation with asurgical microscope.

The color image representation 11 of the capture region captured with acamera is converted into a logarithmic intensity image 14. The intensityimage 14 is subsequently processed, in particular in a plurality ofiterations, in five measures 201 to 205 with the evaluation device 4 ofthe surgical microscope 1 (FIG. 1).

A two-dimensional spatial filter in the form of a two-dimensionalGaussian spatial filter is applied to the intensity image 14 in measure201. In particular, this is implemented by an iterative application tothe intensity image 14. In this case, a filter variable is chosen with alinearly increasing filter variable (σ_(i)=iσ), in particular dependingon the iteration, where i is the current iteration of a total of kiterations.

A value of a gradient of the intensity in the intensity image 14 isdetermined in measure 202. The value of the gradient is determinedpixel-by-pixel in particular and is required for the edge stop function.An example of a measure for the value of the gradient was alreadyspecified above.

A weighted edge stop function is defined in measure 203. By way of theedge stop function it is possible, in particular, to reduce or minimizeexcessive smoothing effects at the edges as a result of filtering withthe Gaussian spatial filter. Whenever the calculated gradient is largerthan a predetermined value, a weighting parameter for a summation inmeasure 204 is set to zero. An example of an edge stop function hasalready been specified above.

In measure 204, a filtered image representation is produced from theoriginal intensity image 14 and the intensity image 14 that has beenfiltered with the Gaussian spatial filter, by virtue of the originalintensity image 14 and the intensity image 14 that has been filteredwith the Gaussian spatial filter being mixed, with weighting, to form asummation image representation 11 s, the weighting parameter for eachpixel being specified by the edge stop function. An example of theweighted summation for mixing purposes has already been specified above.

In measure 205, the detailed image 13 is generated from the intensityimage 14 and the summation image representation 11 s produced in measure204. To this end, the summation image representation 11 s is subtractedpixel by pixel from the intensity image 14.

In FIG. 4, this is indicated for a plurality of iterations from 1 to k.

The final detailed image 13 is converted into an absolute detailed image18. In particular, this is implemented so as to remove negative valueswhich may have originated from the preceding subtraction. In particular,this procedure ensures that no important information is lost when thedetailed image 13 (or the absolute detailed image 18) is subsequentlymixed with the other image representations 11, 12.

An exemplary MATLAB code for generating the absolute detailed image 18is shown below:

img_detail=img_Y_log−img_baseLayer;

img_detail=img_detail+(abs(min(min(img_detail))/10));

This absolute detailed image 18 is subsequently mixed with thefluorescence image representation 12 so that an enriched fluorescenceimage representation 12 a arises, the latter subsequently being mixedwith the color image representation 11 (not shown, see FIG. 2).

Alternatively, provision can also be made for the absolute detailedimage 18 to be converted back into the original color space (e.g., RGB)such that a color detailed image 18 f arises, in order to subsequentlymix the latter with the color image representation 11 (not shown).

FIG. 5 shows a schematic illustration for elucidating a processing chainin a surgical microscope and the integration of the method described inthis disclosure for providing an image representation with a surgicalmicroscope into the processing chain. Not all measures need tonecessarily be carried out in this context; therefore, some of themeasures may also be optional.

Geometric distortion corrections can be carried out on the capturedfluorescence image representation 12 and on the captured color imagerepresentation 11 in measures 300 and 400. In particular, this isadvantageous if an optical arrangement of the camera 2 and thefluorescence camera 3 does not yet or does not completely undertake sucha correction.

Shading corrections may be undertaken in measures 301 and 401.

An intensity calibration may be carried out in a measure 302.

An image improvement and/or image enrichment can be carried out in ameasure 303. By way of example, this measure 303 may include a measure303 a for improving the contrast. Further, as measure 303 b, thismeasure 303 may include the method described in this disclosure forproviding an image representation with a surgical microscope, within thescope of which a detailed image is produced, the latter being used toenrich the captured fluorescence image representation 12 to form anenriched fluorescence image representation 12 a. The fluorescence imagerepresentation may be color coded in a measure 303 c, as a result ofwhich the information from the fluorescence image representation 12 isbetter identifiable in a mixed image representation 20. To this end,there may also be color coding on the basis of the intensity values(“pseudo-coloring”) in a measure 303 d. A measure 303 e includes theidentification and marking of a boundary of a tumor.

The fluorescence image representation 12 processed in this way and thecolor image representation 11 processed in this way are mixed in ameasure 304 to form a mixed image representation 20 (or 201 and 20 r).If this is implemented for two channels (right and left), that is to saystereoscopically, it is possible to produce and provide athree-dimensional mixed image representation 20-3D.

An advantage of the method described in this disclosure and of thesurgical microscope described lies in an improved provision of a mixedimage representation, in which details from a captured color imagerepresentation still are easily recognizable, even after mixing with acaptured fluorescence image representation.

It is understood that the foregoing description is that of the exemplaryembodiments of the disclosure and that various changes and modificationsmay be made thereto without departing from the spirit and scope of thedisclosure as defined in the appended claims.

LIST OF REFERENCE NUMERALS

-   1 Surgical microscope-   2 Camera-   2 l Camera (left channel)-   2 r Camera (right channel)-   3 Fluorescence camera-   3 l Fluorescence camera (left channel)-   3 r Fluorescence camera (right channel)-   4 Evaluation device-   5 Computing device-   6 Memory-   7 Interface-   8 Display device-   10 Capture region-   11 Color image representation-   11 s Summation image representation-   12 Fluorescence image representation-   12 a Enriched fluorescence image representation-   13 Detailed image-   14 Intensity image-   15 Spatial filter-   16 Edge stop function-   17 Weighting parameter-   18 Absolute detailed image-   18 f Color detailed image-   20 Mixed image representation-   20 l Mixed image representation (left channel)-   20 r Mixed image representation (right channel)-   20-3D Three-dimensional mixed image representation-   21 Image signal-   100-104 Measures-   201-205 Measures-   300-304 Measures-   400-401 Measures-   i Iteration-   k Number of iterations

What is claimed is:
 1. A method for providing an image representationwith a surgical microscope, the method comprising: capturing a colorimage representation of a capture region with a camera; capturing afluorescence image representation of the capture region with afluorescence camera; generating a detailed image from the color imagerepresentation with a spatial filter and an edge stop function; mixingthe color image representation, the fluorescence image representation,and the detailed image to form a mixed image representation; andproviding an image signal which encodes the mixed image representation.2. The method as claimed in claim 1, further comprising: generating anintensity image from the color image representation, and wherein thedetailed image is generated from the intensity image.
 3. The method asclaimed in claim 2, wherein the intensity image is at least one of (a)generated, and (b) processed on a logarithmic scale.
 4. The method asclaimed in claim 1, wherein the spatial filter is a two-dimensionalGaussian spatial filter.
 5. The method as claimed in claim 2, wherein aposition-related value of a gradient of an intensity in the generatedintensity image is provided as an input parameter of the edge stopfunction.
 6. The method as claimed in claim 5, wherein the detailedimage is generated by a weighted summation of the color imagerepresentation filtered with the spatial filter and the color imagerepresentation, followed by a subtraction of a summation imagerepresentation from the color image representation, with an output valueof the edge stop function being provided as weighting parameter.
 7. Themethod as claimed in claim 1, wherein generating of the detailed imageis repeated iteratively with altered parameters.
 8. The method asclaimed in claim 1, wherein, during mixing, the detailed image is mixedwith the fluorescence image representation and a resultant enrichedfluorescence image representation is mixed with the color imagerepresentation.
 9. The method as claimed in claim 1, further comprising:carrying out at least one of (a) a geometric distortion correction, and(b) a shading correction for the camera and the fluorescence camera. 10.The method as claimed in claim 1, wherein the method is carried out onat least one of (a) a stereoscopic camera, and (b) a stereoscopicfluorescence camera.
 11. A method for providing an image representationfor a surgical microscope, the method comprising: receiving a colorimage representation of a capture region captured with a camera;receiving a fluorescence image representation of the capture regioncaptured with a fluorescence camera; generating a detailed image fromthe color image representation with a spatial filter and an edge stopfunction; mixing the color image representation, the fluorescence imagerepresentation, and the detailed image to form a mixed imagerepresentation; and providing an image signal which encodes the mixedimage representation.
 12. A surgical microscope, comprising: a cameraconfigured to capture a color image representation of a capture region;a fluorescence camera configured to capture a fluorescence imagerepresentation of the capture region; an evaluation device configuredto: extract a detailed image from the color image representation with aspatial filter and an edge stop function; mix the color imagerepresentation, the fluorescence image representation, and the extracteddetailed image to form a mixed image representation, and provide animage signal which encodes the mixed image representation.